Search for: All records

The space physics community continues to grow and become both more interdisciplinary and more intertwined with commercial and government operations. This has created a need for a framework to easily identify what projects can be used for specific applications and how close the tool is to routine autonomous or on-demand implementation and operation. We propose the Application Usability Level (AUL) framework and publicizing AULs to help the community quantify the progress of successful applications, metrics, and validation efforts. This framework will also aid the scientific community by supplying the type of information needed to build off of previously published work and publicizing the applications and requirements needed by the user communities. In this paper, we define the AUL framework, outline the milestones required for progression to higher AULs, and provide example projects utilizing the AUL framework. This work has been completed as part of the activities of the Assessment of Understanding and Quantifying Progress working group which is part of the International Forum for Space Weather Capabilities Assessment. 

Abstract As part of its International Capabilities Assessment effort, the Community Coordinated Modeling Center initiated several working teams, one of which is focused on the validation of models and methods for determining auroral electrodynamic parameters, including particle precipitation, conductivities, electric fields, neutral density and winds, currents, Joule heating, auroral boundaries, and ion outflow. Auroral electrodynamic properties are needed as input to space weather models, to test and validate the accuracy of physical models, and to provide needed information for space weather customers and researchers. The working team developed a process for validating auroral electrodynamic quantities that begins with the selection of a set of events, followed by construction of ground truth databases using all available data and assimilative data analysis techniques. Using optimized, predefined metrics, the ground truth data for selected events can be used to assess model performance and improvement over time. The availability of global observations and sophisticated data assimilation techniques provides the means to create accurate ground truth databases routinely and accurately. 

Abstract Geomagnetic indices are convenient quantities that distill the complicated physics of some region or aspect of near‐Earth space into a single parameter. Most of the best‐known indices are calculated from ground‐based magnetometer data sets, such as Dst, SYM‐H, Kp, AE, AL, and PC. Many models have been created that predict the values of these indices, often using solar wind measurements upstream from Earth as the input variables to the calculation. This document reviews the current state of models that predict geomagnetic indices and the methods used to assess their ability to reproduce the target index time series. These existing methods are synthesized into a baseline collection of metrics for benchmarking a new or updated geomagnetic index prediction model. These methods fall into two categories: (1) fit performance metrics such as root‐mean‐square error and mean absolute error that are applied to a time series comparison of model output and observations and (2) event detection performance metrics such as Heidke Skill Score and probability of detection that are derived from a contingency table that compares model and observation values exceeding (or not) a threshold value. A few examples of codes being used with this set of metrics are presented, and other aspects of metrics assessment best practices, limitations, and uncertainties are discussed, including several caveats to consider when using geomagnetic indices.